Skip to content

[python] Simplify LeRobot metadata tables and training tags - #9676

Merged
JingsongLi merged 1 commit into
masterfrom
codex/lerobot-metadata-tags
Sep 8, 2026
Merged

[python] Simplify LeRobot metadata tables and training tags#9676
JingsongLi merged 1 commit into
masterfrom
codex/lerobot-metadata-tags

Conversation

@YannByron

@YannByron YannByron commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Purpose

Decouple LeRobot ingestion from dataset version publication. Store dataset info and optional statistics in companion tables with key STRING, value STRING, using JSON-encoded values to preserve nested metadata without a fixed schema.

load_from_lerobot returns None on success and accepts an optional tag_name. create_lerobot_tag pins the current snapshots of all recorded components under a shared name for training, checks conflicts before creating tags, and supports retries while snapshots remain unchanged.

Cross-table tagging is not atomic: callers must pause group writes until the call succeeds and read every component through the same tag. Documentation and regression tests cover this contract.

Tests

  • LeRobot import suite: 54 tests and 46 subtests passed, including JSON round-trips, optional metadata, stable tagged reads after appends, conflicts, missing components, and partial tag failure/retry.
  • Multimodal table and filesystem catalog tag suites: 103 passed, 2 skipped because Ray was unavailable.
  • Flake8 on changed Python files and git diff --check passed.

LeRobot tests ran with Python 3.12, LeRobot 0.4.4, PyArrow 19.0.1, and NumPy 2.3.5. NumPy 2.5 fails in the existing SDK fixture's array-to-scalar conversion; project dependencies are unchanged.

@YannByron
YannByron force-pushed the codex/lerobot-metadata-tags branch from ee7d782 to d9d4e82 Compare September 7, 2026 14:40
Comment thread docs/docs/pypaimon/multimodal-api.mdx Outdated
@JingsongLi

Copy link
Copy Markdown
Contributor

There are no other issues besides those mentioned in the comments above.

Store info and stats in companion tables with JSON string values, and
support optional named tags for imported or current component snapshots.

Co-Authored-By: Codex <noreply@anthropic.com>
AI-Model: gpt-6
Co-Authored-By: Codex <noreply@ai-tool.com>
AI-Contributed/Feature: 22/22
AI-Contributed/UT: 38/38
@YannByron
YannByron force-pushed the codex/lerobot-metadata-tags branch from d9d4e82 to 12e030f Compare September 8, 2026 06:12

@JingsongLi JingsongLi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi
JingsongLi merged commit c947e5c into master Sep 8, 2026
17 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants